You cannot delete a layer that contains objects. (But holding the option key, while clicking the delete button, will PERMANENTLY delete all objects in the layer).